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TOWARD  A NONPARAMETRIC  COVARIANCE  ANALYSIS 
OF  A WEATHER  MODIFICATION  EXPERIMENT 


*A  further  analysis  of  the  1967-74  Santa  Barbara  weather  modification  experi- 
ment data  Is  In  progress  In  the  Department  of  Statistics,  Florida  State  University. 
The  present  report  provides  a nonparametric  covariance  analysis,  of  limited  scope, 
of  the  Phase  I experimental  data.  The  findings  tend  to  confirm,  qualitatively , 
the  North  American  Weather  Consultants  conclusions  of  a demonstrable  seeding 
effect  under  selected  conditions  of  cloud  instability  and  cloud  temperature. 
Quantltlvely,  however,  the  present  findings  are  more  restrained.  The  analysis 
tends  to  suggest  the  need  for  Improved  experimental  designs. 


X 
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1.  Introduction.  A series  of  weather  modification  experiments  was  con- 

“ 1 ) 

■ : . ••••  h 

ducted  from  1967-74  by  North  American  Weather  Consultants  (NAWC)  under  the 
sponsorship  of  the  Naval  Weapons  Center,  China  Lake,  California.  The  experi- 
ments were  carried  out  at  Santa  Barbara,  California,  with  the  objective  of 
evaluating  the  effectiveness  of  cloud  seeding  and  seeding  methods  in  West 
Coast  cyclonic  winter  and  spring  storms.  During  Phase  I,  covering  the  1967-68 
through  the  1970-71  seasons,  the  primary  seeding  mode  was  ground  based.  During 
Phase  II,  covering  the  1971-72  through  the  1973-74  seasons,  the  seeding  mode 
was  aerial.  An  overview  of  the  experiments  and  NAWC 'a  data  analysis  is  pro- 
vided in  the  fir.nl  report  [3], 

In  [3]  the  data  was  organized  and  analyzed  on  a 8tation-by-8tation  basis. 
There  were  two  networks  of  ralngage  stations,  those  in  the  "target"  area  and 
those  in  the  "control"  area.  During  Phase  I,  the  experimental  units  consisted 
of  107  "seedable"  convection  bands,  each  of  which  passed  through  both  the 
control  and  target  areas.  By  a random  process,  56  experimental  units  were 
selected  for  actual  seeding.  For  each  ralngage  station,  the  ralngage  preci- 
pitation measurements  for  the  56  seeded  bands  were  averaged  and  likewise  the 
average  for  the  51  nonseeded  bands  wrs  formed,  and  the  ratio  of  these  two 
averages  was  taken  as  a basic  quantitative  variable  measuring  seeding  effect. 
The  effect  of  seeding  was  then  assessed  by  comparing  the  ratios  for  target 
network  stations  with  those  for  the  control  area.  Table  1 gives  the  distri- 
bution of  stations  by  magnitude  of  ratio  (seeded  band  average/nonseeded  band 
average),  for  control  and  target  areas. 
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Table  1.  Number  of  Stations,  by  Magnitude  of  Precipitation  Ratio,  by  Control 
and  Target  Areas. 


Magnitude  Number  of  Stations 

of  Ratio Control  Area Target  Area 


Under  1.000 

6 

1.000-1.195 

20 

1.200-1.395 

10 

1.400-1.595 

3 

Over  1.600 

0 

1 

6 

20 

22 

12 


Total 


39 


61 


Source:  Derived  from  [3],  Appendix  II.  (Also  produced  in  [4],  page  10) 


The  target  area  ratios  appear  to  be  stochastically  larger  than  the  control 
area  ratios.  Indicating  that  seeding  tends  to  yield  an  Increase  In  precipitation. 
However,  It  Is  difficult  to  test  the  statistical  significance  of  such  a result, 
due  to  the  stochastic  dependence  among  the  ratios  (the  ratios,  being  based  on 
averages  taken  over  the  same  experimental  units,  would  be  highly  correlated 
for  stations  In  close  proximity).  Moreover,  the  control  area  ratios  exceed 
the  value  1 with  peculiarly  high  frequency,  thus  making  it  difficult  to  inter- 
pret the  meaning  of  a high  value  for  such  a precipitation  ratio.  Nor  does 
this  approach  lead  to  a quantitative  determination  of  the  seeding  effect.  In 
the  sense  of  a measure  of  the  actual  Increase  in  precipitation  produced  over 
the  target  area  as  a whole.  Because  of  issues  such  as  these,  it  appears 
reasonable  to  explore  other  approaches  toward  analysis  of  the  data,  before 
concluding  that  "beyond  any  reasonable  doubt  the  seeding  was  successful  In 
increasing  rainfall  In  a prespecified  area"  [1]. 


j 


A further  analysis  of  the  Santa  Barbara  data,  for  Phase  I,  has  been 
launched  in  the  Department  of  Statistics,  Florida  State  University,  under  the 
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direction  of  Ralph  A.  Bradley.  The  data  Is  being  organized  and  analyzed  by 
individual  bands  (experimental  units).  As  an  initial  step,  a summarization 
of  the  precipitation  data  by  response  surface  methods  has  been  carried  out  and 
reported  by  Bradley,  Srlvastava  and  Lanzdorf  [4].  For  each  experimental  unit, 
separately  over  the  control  and  target  areas,  response  surfaces  are  fitted 
to  the  raingage  measurements.  These  surfaces  provide  a framework  for  develop- 
ment of  a variety  of  analyses. 


The  present  paper  augments  [4].  The  data  is  again  organized  by  individual 
bands,  but  instead  of  a response  surface  approach  the  quantity  of  rainfall 
over  a network  of  stations  is  represented  by  the  simple  average  of  raingage 
measurements.  (It  is  felt,  on  the  basis  of  some  casual  theoretical  reflections, 
that  these  traditionally  used  measures  are  highly  correlated  with  volumes 
obtained  by  integrating  under  surfaces.)  Further,  the  experimental  units  are 
grouped  not  only  by  "seeded"  or  "nonseeded,"  but  also  by  covariates  corres- 
ponding to  (1)  the  stability  of  the  convection  band,  (2)  the  500  mb  temperature 
of  the  convection  band.  This  provides  a framework  for  a nonparametrlc  covariance 


analysis  of  the  data.  Section  2 presents  the  relevant  tabulations  and  compu- 
tations; Section  3 presents  conclusions,  findings  and  recommendations. 


2.  Tabulations  and  computations.  Table  2 provides  a classification  of 
the  107  Phase  I experimental  units  by  "seeded"  or  "nonseeded,"  by  air  mass 
stability,  and  by  500  mb  cloud  temperature.  Following  [1],  the  air  mass  sta- 
bility categories  are  taken  to  be 

UH:  Unstable,  High  Convective  Instability  Base  (CIB) 

UL:  Unstable,  Low  CIB 


S:  Stable 
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and  the  500  mb  temperature  categories  are  taken  to  be 
Cold:  -22.5°C.  or  below 
Medium:  Above  -22.5°C.,  below  -17.5°C. 

Warm:  -17.5°C.  or  above. 

Refer  to  [1]  for  discussion  of  the  meteorological  significance  of  these  cate- 
gories . 

Table  2.  Number  of  Experimental  Units,  by  Seeded  or  Nonseeded,  by  Stability 
Category  and  by  Temperature  Category. 


Temperature  Category 

5 tauXXXCy 

Category 

Cold 

Medium 

Warm 

Total 

S NS 

S NS 

S NS 

S NS 

UH 

1 5 

7 6 

3 1 

11  12 

UL 

16  12 

14  11 

10  11 

40  34 

S 

0 0 

5 1 

0 4 

5 5 

Total 

17  17 

26  18 

13  16 

56  51 

Note  the  imbalances  within 

a)  the  UH  row 

b)  the  S row 

c)  the  Medium  column 

d)  the  Warm  column. 

These  cause  a confounding  of  certain  comparisons  between  seeding  and  nonseeding, 
for  example  the  comparison  of  S versus  NS  for  the  UH  category. 


It  is  thus 
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desirable  to  analyze  the  data  separately  within  the  cells  of  Table  2.  Corre- 
sponding to  a labelling  of  the  cells  according  to  the  pattern 


12  3 


Tables  A1-A9  (A7  omitted)  provide  for  each  cell  the  average  precipitation 
for  control  stations,  the  average  precipitation  for  target  stations,  and  the 
ratio  of  target  average  to  control  average,  by  bands  classified  seeded  or  non- 
seeded.  (These  tables  are  derived  from  the  sources  [1]  and  [2].) 


Table  Al.  For  Phase  I Bands  in  "UH"  Stability  Category  and  "Cold"  Temperature 
Category:  Average  Precipitation  for  Control  Stations,  Average 
Precipitation  for  Target  Stations,  and  Ratio  of  Target  Average  to 
Control  Average,  by  Bands  Classified  Seeded  or  Nonseeded. 


Nonseeded  Bands 

Seeded  Bands 

Season 

and 

Band  No. 

Avg.  Preclp., 
Control  Stat. 
X 

Avg.  Precip., 
Target  Stat. 

Y 

Precip . 

Ratio 

R - Y/X 

Season 

and 

Band  No. 

Avg.  Precip., 
Control  Stat. 
X 

Avg.  Precip., 
Target  Stat. 

Y 

Precip. 

Ratio 

R - Y/X 

68-69, #31 

.183 

.228 

1.25 

68-69, #34 

.737 

1.266 

1.72 

#33 

.115 

.106 

.92 

#41 

.522 

.410 

.79 

70-71, #12 

.267 

.267 

1.00 

#13 

.043 

.108 

2.51 

b 
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Table  A2.  For  Phase  I Bands  In  "UH"  Stability  Category  and  "Medium"  Temperature 
Category:  Average  Precipitation  for  Control  Stations,  Average  Pre- 
cipitation for  Target  Stations,  and  Ratio  of  Target  Average  to  Control 
Average,  by  Bands  Classified  Seeded  or  Nonseeded. 


Nonseeded  Bands 


Season  Avg.  Preclp.,  Avg.  Precip.,  Preclp. 
and  Control  Stat.  Target  Stat.  Ratio 
Band  No.  X Y R - Y/X 


Seeded  Bands 


Season  Avg.  Preclp.,  Avg.  Preclp.,  Precip. 
and  Control  Stat.  Target  Stat.  Ratio 
Band  No.  X Y R - Y/X 


#4 

.156 

.072 

.46 

#9 

.093 

.048 

.52 

#25 

.025 

.062 

2.48 

#37 

.567 

.867 

1.53 

#39 

.147 

.114 

.78 

,#3 

.052 

.070 

1.35 

#8 

#24 

#26 

#28 

#30 

69-70, #8 


xdDie  A3.  For  Phase  I Bands  In  "UH"  Stability  Category  and  "Warm"  Temperature 
Category:  Average  Precipitation  for  Control  Stations,  Average 
Precipitation  for  Target  Stations,  and  Ratio  of  Target  Average 
to  Control  Average,  by  Bands  Classified  Seeded  or  Nonseeded. 
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Table  A4.  For  Phase  I Bands  In  "DL"  Stability  Category  and  "Cold”  Temperature 
Category:  Average  Precipitation  for  Control  Stations,  Average  Pre- 
cipitation for  Target  Stations,  and  Ratio  of  Target  Average  to 
Control  Average,  by  Bands  Classified  Seeded  or  Nonseeded . 


Nonseeded  Bands 

Seeded 

Bands 

Season 

Avg.  Precip., 

Avg.  Precip., 

Precip . 

Season 

Avg.  Precip., 

Avg.  Precip., 

Precip . 

and 

Control  Stat. 

Target  Stat. 

Ratio 

and 

Control  Stat. 

Target  Stat. 

Ratio 

Band  No. 

X 

Y 

R - Y/X 

Band  No. 

X 

Y 

R - Y/X 

67-68,11 

.177 

.091 

.51 

67-68, #4 

.078 

.049 

.63 

#2 

.207 

.203 

.98 

#7 

.097 

.080 

.82 

#3 

.083 

.168 

2.02 

#16 

.150 

.128 

.85 

#22 

.105 

.068 

.65 

#19 

.543 

.698 

1.28 

68-69, #32 

.400 

.554 

1.38 

#20 

.027 

.060 

2.22 

69-70, #14 

.237 

.593 

2.50 

#21 

.087 

.086 

.99 

#17 

.300 

.221 

.74 

69-70, #18 

.100 

.199 

1.99 

#20 

.120 

.123 

1.02 

#19 

.162 

.091 

.56 

70-71, #8 

.240 

.252 

1.05 

#21 

.093 

.242 

2.60 

#14 

.720 

.850 

1.18 

#22 

.813 

1.016 

1.25 

#17 

.078 

.056 

.72 

70-71, #9 

.052 

.047 

.90 

#19 

.023 

.085 

3.69 

#10 

.073 

.155 

2.12 

#11 

.267 

.478 

1.79 

#15 

.085 

.115 

1.35 

#16 

.178 

.074 

.42 

#18 

.158 

.210 

1.33 

Table  A5.  For  Phase  1 Bands  in  "UL"  Stability  Category  and  "Medium"  Temperature 
Category:  Average  Precipitation  for  Control  Stations,  Average 
Precipitation  for  Target  Stations,  and  Ratio  of  Target  Average  to 
Control  Average,  by  Bands  Classified  Seeded  or  Nonseeded. 


Nonseede 

d Band 8 

Seeded 

Season 

Avg.  Precip., 

Avg.  Precip., 

Precip. 

Season 

Avg.  Precip., 

and 

Control  Stat. 

Target  Stat. 

Ratio 

and 

Control  Stat. 

Band  No. 

X 

Y 

R - Y/X 

Band  No. 

X 

67-68, #6 

.092 

.081 

.88 

67-68, #5 

.122 

#9 

.020 

.009 

.45 

#8 

.010 

#18 

.257 

.131 

.51 

#15 

.163 

68-69, #14 

.125 

.060 

.48 

68-69, #23 

.132 

#19 

.085 

.272 

3.28 

69-70, #2 

.363 

#20 

.312 

.670 

2.15 

#5 

.257 

#35 

.432 

.360 

.83 

#12 

.023 

69-70, #1 

.628 

.433 

.69 

#13 

.053 

#6 

.213 

.237 

1.11 

#15 

.513 

#7 

.048 

.027 

.56 

#16 

.405 

70-71, #22 

.358 

.295 

.82 

70-71, #4 

.050 

#6 

.182 

#7 

.114 

HI 

#21 

.340 

L 


_L... 


JLL 
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Table  A6.  For  Phase  I Bands  In  "UL"  Stability  Category  and  "Warn"  Temperature 

• i r 

Category:  Average  Precipitation  for  Control  Stations,  Average 
Precipitation  for  Target  Stations,  and  Ratio  of  Target  Average  to 


Control  Average,  by  Bands  Classified  Seeded  or  Nonseeded. 


1 


Nonseeded 

Bands 

’ . * 

-- 

: { 

Seeded  Bands 

- - ■- 

Season 

Avg.  Precip., 

Avg.  Precip., 

Precip. 

Season 

Avg.  Precip., 

Avg.  Precip., 

Precip . 

and 

Control  Stat. 

Target  Stat. 

Ratio 

and 

Control  Stat. 

Target  Stat. 

Ratio 

Band  No. 

X 

Y 

R - Y/X 

Band  No. 

X 

Y 

R - Y/X 

67-68, #10 

.027 

.029 

1.07 

67-68, #14 

.118 

.087 

.69 

#11 

.165 

.210 

1.27 

68-69, #13 

.165 

.184 

1.12 

#12 

.255 

.275 

1.08 

#16 

.162 

.348 

2.15 

#13 

.063 

.067 

1.06 

#17 

.310 

.469 

1.51 

68-69, #10 

.023 

.035 

1.52 

#18 

.227 

.599 

2.64 

#15 

.297 

.268 

.90 

#22 

.417 

.898 

2.15 

#21 

.535 

.670 

1.25 

69-70, #11 

.148 

.126 

.85 

#27 

.200 

.181 

.90 

70-71, #1 

.358 

.135 

.38 

69-70, #10 

.122 

.120 

.98 

#3 

.168 

.136 

.81 

70-71, #2 
#5 

.152 

.018 

.104 

.085 

.68 

4.72 

1 

#20 

.475 

.525 

1.10 

Table  A8. 
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For  Phase  I Bands  in  "S"  Stability  Category  and  "Medium"  Temperature 


Category:  Average  Precipitation  for  Control  Stations,  Average 
Precipitation  for  Target  Stations,  and  Ratio  of  Target  Average 
to  Control  Average,  by  Bands  Classified  Seeded  or  Nonseeded. 


Nonseeded 

Bands 

Seeded 

Bands 

Season 

Avg.  Precip., 

Avg.  Precip., 

Precip . 

Season 

Avg.  Precip., 

Avg.  Precip., 

Precip. 

and 

Control  Stat. 

Target  Stat. 

Ratio 

and 

Control  Stat. 

Target  Stat. 

Ratio 

Band  No. 

X 

Y 

R - Y/X 

Band  No. 

X 

Y 

R - Y/X 

■ 

68-69, #29 

.062 

.076 

1.22 

67-68, #17 

.870 

.633 

mm 

68-69, #3 

.068 

.084 

mmm 

#38 

.075 

.085 

#40 

.391 

.688 

1.76 

69-70, #4 

.025 

.034 

1.36 

Table  A9.  For  Phase  I Bands  in  "S"  Stability  Category  and  "Warm"  Temperature 


Category:  Average  Precipitation  for  Control  Stations,  Average 
Precipitation  for  Target  Stations,  and  Ratio  of  Target  Average 
to  Control  Average,  by  Bands  Classified  Seeded  or  Nonseeded. 


Nonseeded  Bands 

Seeded  Bands 

Season 

and 

Band  No. 

Avg.  Precip., 
Control  Stat. 
X 

Avg.  Precip., 
Target  Stat. 

Y 

Precip. 

Ratio 

R - Y/X 

Season 

and 

Band  No. 

Avg.  Precip., 
Control  Stat. 
X 

Avg . Precip . , 
Target  Stat. 

Y 

Precip. 

Ratio 

R * Y/X 

68- 69, #1 

#2 

#7 

69- 70, #9 

.282 

.202 

.097 

.177 

1 J 

.63 

.85 

.69 

.69 

A natural  approach  is  to  analyze  the  ratios  B provided  In  Tables  A1-A9. 
However,  a mere  glance  at  Tables  Al,  A3,  A8  and  A9  reveals  In  ^ach  case  an 
Insufficient  amount  of  data  to  yield  any  statistically  significant  findings. 

The  situation  is  only  slightly  better  in  the  case  of  Table  A2,  but  upon  close 
analysis  it  is  found  that  the  null  hypothesis  of  "no  seeding  effect"  cannot 
be  rejected  by  any  reasonable  statistical  test.  This  is  a consequence  of 
the  high  variability,  within  seeded  and  nonseeded  groups,  of  the  ratios  R. 

The  preponderance  of  the  data  falls  in  cells  4,  5 and  6,  and  so  it  might  be 
hoped  that  definitive  conclusions  may  be  reached  via  Tables  A4,  AS,  and  A6. 
However,  the  difficulty  with  Table  A2  persists:  the  variability  of  the  R- 
ratlos  overwhelms  the  possibility  of  testing  for  significant  difference  between 
the  seeded  and  nonseeded  cases.  Table  3 provides  for  each  of  cells  A4,  A5, 
and  A6  the  average  Rg  of  R-values  for  seeded  bands,  the  average 
for  nonseeded  bands,  and  the  estimated  standard  deviation  of  the  difference 

D * *S  ‘ *NS‘ 


R^s  of  R-values 
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Table  3. 


Cell 

Rs 

D 

°D 

A4 

1.32 

1.37 

- .05 

.30 

A5 

2.97 

1.07 

1.90 

1.25 

A6 

1.34 

1.40 

- .06 

.39 

Therefore,  the  R-ratlos  in  Tables  A1-A9  may  be  abandoned.  (The  value  D « 1.90 
for  Table  A5  appears  to  provide  strong  evidence  in  support  of  a seeding  effect, 
until  it  ia  noted  that  one  of  the  contributors  to  Rg  is  the  R-value  18.83  for 
R ■ Y/X  with  a denominator  X close  to  0.)  The  approach  of  looking  at  these 
R-values  would  become  feasible  only  if  a considerably  larger  number  of  experi- 
mental units  were  available. 

In  lieu  of  R-ratlos,  Rt  * Y^/X^,  an  alternative  ratio-type  approach  is 
based  on  the  ratios 
(1)  R - (ZY1)/(EX1), 

taken  separately  over  the  seeded  and  nonseeded  units.  This  reduces  the  excessive 
Influence  of  small  X^-values,  but  it  measures  a less  interesting  parameter. 

Table  4 provides  for  each  of  the  cells  A4,  A5  and  A6  the  ratio  Rg  of  form  (1) 
for  seeded  bands,  the  ratio  RNg  of  form  (1)  for  nonseeded  bands,  and  the  esti- 
mated standard  deviations  of  Rg  and  R^g . 


Table  4. 


11  -r 


, j } 

It  is  evident  that,  as  measured  by  Rg  - R^g,  for,  cells  A5  and  A6  there  is 
a clear-cut  statistically  significant  seeding  effect.  It  is  especially  pro- 
nounced for  A5. 

REMARKS,  (£)  The  control  stations  numbered  6,  the  target  stations  54, 

V i r ... 

' '■  . 

in  the  experiment  as  described  in  [1).  Subsequently, 'data  was  provided 
for  additional  control  stations  and  additional  target  stations.  However, 
a parallel  development  of  Tables  A1-A9  and  Tables  3 and  4 yields  essentially 
similar  results.  Herein  we  have  presented  the  version  suitable  for  direct 
comparison  with  [1]. 

(££)  An  analysis  of  the  R-ratios  with  respect  to  medians  Instead  of 
means,  Rg  and 

(£££)  A number  of  plots  associated  with  Tables  A1-A9  were  constructed 
and  examined.  These  provided  little  additional  insight  of  definitive  nature. 

3.  Comments . The  analysis  of  Section  2 yields  the  following  general 
conclusions : 

(£)  The  data  is  insufficient  to  yield  positive  affirmation  or  rejection 
of  the  "null"  hypothesis,  except  in  certain  cases.  These  cases  correspond 
to  the  "UL"  stability  category,  especially  the  "Medium"  and  "Warm"  temperature 
range. 

(££>  In  the  "UL"  - "Cold"  case,  the  null  hypothesis  appears  well  sub- 
stantiated. 

(£££)  In  the  "UL"  - "Medium"  case,  the  null  hypothesis  appears  very 
strongly  rejected. 

(£u)  In  the  "UL"  - "Warm"  case,  the  null  hypothesis  appears  strongly 


Rgg,  leads  to  essentially  similar  results. 


rejected. 
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While  not  in  severe  contradiction  to  the  findings  in  (1],  [2),  I3J,  the 
above  conclusions  are  considerably  more  restrained.  They  are  solely  quali- 
tative, not  quantitative. 

With  greater  Investment  of  time  and  sophistication,  a more  informative 
version  of  the  nonparametric  covariance  analysis  presented  here  might  be  carried 
out.  However,  the  gain  would  be  slight  compared  to  the  reward  which  might  accrue 
from  an  experimental  design  of  greater  sophistication  and  with  a greater  number 
of  experimental  units.  For  example,  one  might  achieve  greater  wlthln-cells 
balance  between  seeding  and  nonseeding. 
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